Advantages of Di erential Dynamic Programming OverNewton ' s Method for Discrete - Time Optimal Control

Diierential Dynamic Programming (DDP) and stagewise Newton's method are both quadratically convergent algorithms for solving discrete time optimal control problem. Although these two algorithms share many theoretical similarities, they demonstrate signii-cantly diierent numerical performance. In this paper, we will compare and analyze these two algorithms in detail, and derive another quadratically convergent algorithm which is a combination of the DDP algorithm and Newton's method. This new second-order algorithm plays a key role in the explanation of the numerical diierences between the DDP algorithm and Newton's method. The detailed algorithmic and structural diierences for these three algorithms and their impact on numerical performance will be discussed and explored. Two test problems with various dimensions solved by these three algorithms will be presented. One nonlinear test problem demonstrates that the DDP algorithm can be as much as 28 times faster than the stagewise Newton's method. The numerical comparison indicates that the DDP algorithm is numerically superior to the stagewise Newton's method.